Dear Editor: I Write to Congratulate Fisher Et Al for Their Work on the Eye Surgical Skills Assessment Test (essat) for Ophthalmology Residency

Dear Editor: I write to congratulate Fisher et al for their work on the Eye Surgical Skills Assessment Test (ESSAT) for ophthalmology residency. The determination of the validity of tools for assessing resident competency is a crucial part of the Accreditation Council for Graduate Medical Education outcomes project over time. It might be worthwhile for the readership to understand how different types of validity might be tested for a tool like the ESSAT. The authors reported that the ESSAT had content and face validity. Content validity is a measure of the alignment of the tool content with the performance domain (i.e., technical skills or surgical knowledge or competency) that the tool is designed to assess. Face validity is a type of commonsense or “reasonable man” measure that can be judged by an external stakeholder (e.g., the lay patient, learners themselves, the public). All of these validity types help to answer the question “Is this tool measuring what we think it is measuring?” As noted by the authors, there are other forms of validity that might be useful to look at with the ESSAT, and I hope the authors might comment on their experience with or plans for such validity testing. For example, curricular validity is the degree that the tool content matches the objectives of the authors’ institutional or departmental surgical curriculum. Criterion-related validity examines the correlation, if any, between the tool scores and some other desired learner outcome. For example, is there a correlation of the ESSAT with the global evaluation forms by faculty of individual resident surgeons, self-assessments by resident surgeons of performance, or surgical outcomes (e.g., vitreous loss rate, visual outcome)? Predictive validity reflects the ability of the tool to predict future resident performance. Concurrent validity is determined when the results with 2 tools used on the same learner at the same time (i.e., concurrently) show the same or similar results (e.g., performance on the ESSAT vs. other assessments in the wet laboratory, simulated or live surgery experience). In a similar manner, convergent validity is a measure of how closely 2 tools that measure similar or closely related skills correlate, and discriminant validity is a measure showing that 2 tests that do not measure closely related skills do not correlate strongly. One easy question that gets to the heart of the testing, “Does an expert (e.g., faculty member) perform better than a proficient learner (e.g., fellow, third-year resident) and better still than a novice (e.g., first-year resident)?” I believe that the authors have done an excellent job of establishing the face and content validity for the ESSAT. Future work might focus on establishing criterion, predictive, concurrent, convergent, and discriminant validity.